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(54) Input device simulating touch screen 

(57) An input device comprises a two-dimensional surface 12 capable of presenting predefined regions 13 
and 14 to a user, a video camera 6 directed at said surface for creating real-time images thereof, a device using 
a Rybak method to analyse the real-time images to identify a human hand and therefore decide which 
predefined region is being pointed to by the user, and a device for outputting a signal dependent on the 
identified region pointed to by the user. The input device can act as an alternative pointing device for a 
computer, for example. 
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At least one drawing originally filed was informal and the print reproduced here is taken from a later filed formal 
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2315859 

" 1 " 

INPUT DEVICE SIMULATING TOUCH SCREEN 
This invention relates to a input device, and more 
particularly an input device capable of simulating a touch 
screen. 

Some means must be found for users to provide input to 
computers. The keyboard is the most obvious means, but 
is not always practical because it requires typing ability 
and is not well suited to situations where rapid input is 
required. The touch screen has been developed as one 
solution to this problem. This allows a user to touch a 
part of a screen to indicate a certain input. For 
example, when moving through menus, the user can touch 
certain portions of the screen to select desired menu 
choices . 

The problem with touch screens is that they are expensive 
to produce and also they only work on cathode ray tubes. 
There are situations where "touch screen" capability is 
desirable on static devices, for example posters and the 
like. 

An object of the invention is to alleviate this problem. 

According to the present invention there is provided an 
input device comprising a two-dimensional surface capable 
of presenting predefined regions to a user, a video camera 
directed at said two-dimensional surface for creating 
real-time images thereof, means for analyzing said real- 
time images to identify which pre-defined region is being 
pointed to by a user, and means for outputting a single 
dependent on the identified region pointed to by the user. 

The device preferably includes a pattern recognition 
device capable of recognising the form of a human hand, 



- 2 - 

for example using a modified Rybak method. 

The two-dimensional surface can be a computer screen or 
alternatively a static display, such as a poster or the 
like. 

The invention will now be described in more detail, by way 
of example, only with reference to the accompanying 
drawings, in which: - 

Figure 1 is a block diagram of an input device in 
accordance with the invention,- 

Figure 2 is a diagrammatic view of one embodiment of the 
15 invention- 
Figure 3 illustrates the process for determining the 
region pointed to; 
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Figure 4 is a block diagram of a system in accordance with 
the invention; 

Figure 5 is a more diagrammatic view of an alternative 
arrangement ; and 

Figure 6 is a block diagram of the arrangement shown in 
Figure 5. 

Referring to Figure 1, a central processing unit 1, for 
example an Intel Pentium microprocessor, is connected to 
a bus 2, to which is attached a random access memory 3, 
an output 4 for connection to a device to be controlled,' 
for example a computer (not shown) , and a camera interface 
5 . 
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A still -frame video camera 6 is connected to the camera 
interface 5. The camera 6 is fixedly mounted so as to 
face the computer screen 7 and generate a still -frame 
video images thereof in real time. The camera interface 
5 5 digitizes the output of the camera 6 for connection to 

the bus 2. The images are stored in the RAM 3 as data. 

7 

In use, the video camera 5 continually generates a series 
10 of still frame images of the computer screen 7. 

The CPU 7, under instructions from a program also stored 
in RAM 3 # continually analyzes the stored digital images 
to look for the shape of a human hand using known pattern 
15 recognition techniques, such as a modified Rybak method. 

When a human hand is detected, the CPU 3 then determines 
its terminal location relative to the borders of the image 
and matches this location with predefined areas on the 
image. The CPU 1 then instructs the output device 4 to 

2 0 generate a signal identifying which predefined area is 

being pointed to by the identified hand. This output 
signal can then be passed directly to a computer as X-Y 
co-ordinates in the same way as a signal from a mouse or 
other conventional pointing device. 

25 

Figure 2 shows an alternative embodiment wherein a camera 
6 is directed at a billboard 12 having two well-defined 
regions one representing a graphic 13 and the other text 
14 . As shown in Figure 3 , the camera 6 forms a clean 

3 0 image 15 and a current image 16, which includes the hand 

17 of a user. The system subtracts the current image from 
the clean image to leave only the hand 17. The terminal 
point 18 of the difference image, here point (10 0, 75) , 
is taken to be the point indicated 19, and the system is 
35 . calibrated 19 to know that this relates to a particular 
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predefined region, in this case the graph portion 13 of 
the image . 



Figure 4 illustrates a device for implementing this 
process. The output of the camera 6 is sent to a clean 
image buffer 21 and a current image buffer 22 . The 
outputs of the buffers 21, 22 are subtracted in subtractor 
circuit 20 and passed to a device 12, which implements a 
modified Rybak method to recognize the shape of the human 
hand. The Rybak method, described in a paper entitled 
"BMV: Behavioural Model of Active Visual Perception and 
Invariant Image Recognition" by I. A. Rybak, A.V. Golovan, 
V.I. Gusakova, N.A. Shetsova and L.N. Podladchikovs 
located on the World Wide Web at http://www.voicenet.com/- 
rybak/vnc.html, permits the recognition of certain shapes, 
such as the form of a pointing hand. 

Figure 5 shows a system for monitoring a computer screen, 
which can be used as an input device for the computer. 
The system could, for example, be used in public places 
such as airports where users are require to make menu 
choices. The camera 6, which is fixed in position, is 
directed onto a computer monitor 31 by means of a mirror 
30 . 



First a "snapshot" of the computer screen is taken through 
the operating system of the computer to create a computer 
image 32, referred to as the OS image. This is done 
automatically and can be easily achieved using the built- 
in capability of the operating system. For example, in 
the case of the Windows operating system, pressing the key 
Print Sera causes a bitmap image of the screen to be 
passed the Windows clipboard, and it is a simple matter 
to achieve this internally under program control. 



Next a reference image, which will be referred to as the 
camera image 33, is taken of the clean screen with the 
camera 6. A translation function 34 is then derived to 
translate the camera image to the OS image. This is 
referred to as the delta function A. 

The same procedure is repeated in the presence of an 
object such as a hand 34 and the delta function 35 of the 
current image is formed, which is called the TRANSLATED 
image 36. The TRANSLATED image is then subtracted from 
the current OS image 37 to form a STRIPPED image 38, which 
contains only the difference between the calibration image 
and the present image . 

This procedure makes feature recognition easier using the 
modified Rybak method referred to above. It can also be 
used in conjunction with the other examples referred to 
herein. 

The modified Rybak method is used to determine where the 
hand is pointing to 39, and this point is referred to as 
the HOT SPOT. The X, Y co-ordinates of the HOT SPOT are 
then determined and these are sent to the operating system 
as a mouse position 40 . The current image 34 can be taken 
as often as necessary. 

Figure 6 shows an arrangement for implement ing the above 
method. The snapshot of the screen created by the 
operating system 50 is stored in buffer 41 and that by the 
camera 6 stored in buffer 42 . The delta function is 
created in block 43 and passed through block 44 to 
subtractor 45 where it is subtracted from the OS image 
create the STRIPPED image, which is passed to the modified 
Rybak processing unit 46. Unit 47 derives the.X,Y co- 
ordinates of the hot spot and passes these to the 
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operating system of the computer as if they were a mouse 
input 48. 

The invention thus permits a computer screen or other 
display device, including passive display devices, to be 
given a "touch screen" capability in a particularly 
economical manner. 
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CLAIMS 

1. An input device characterized in that it comprises 
a two-dimensional surface capable of presenting predefined 
regions to a user, a video camera directed at said two- 
dimensional surface for creating real-time images thereof, 
means for analyzing said real-time images to identify 
which pre-defined region is being pointed to by a user, 
and means for output ting a signal dependent on the 
identified region pointed to by the user. 

2. An input device as claimed in claim 2, characterized 
in that it further comprises pattern recognition means for 
recognizing the form of a human hand in said real-time 
image, and said analyzing means generating said output 
signal according to the location of said hand form on said 
two-dimensional surface . 

3. An input device as claimed in claim 2, characterized 
in that said recognizing means uses a modified of the 
Rybak method of pattern recognition. 

4. An input device as claimed in any one of claims 1 to 

3, characterized in that it further comprises means for 
storing a clean image of said two-dimensional surface, and 
wherein said analyzing means subtracts a real-time image 
from said stored clean image to create a difference image 
which is analyzed to find the location of said hand form 
relative to said clean image. 

5. An input device as claimed in any one of claims 1 to 

4, characterized in that said two-dimensional surface is 
a computer screen and said analyzing means subtracts said 
real time image from a snapshot image produced by an 
operating system thereof to produce a delta function which 
translates said current image to the operating system 



image, and said analyzing means compares a real-time image 
translated with said delta function with the operating 
system image to obtain a stripped image that said 
recognition means uses to identify the shape of a human 
hand . 

6. An input device as claimed in any one of claims 1 to 

5, characterized in that said output signal is operative 
to control the operation of a computer . 

7. An input device as claimed in any one of claims l to 

6, characterized in that said video camera is a still - 
frame video camera* 

8. An input device substantially as herein described, 
with reference to the accompanying drawings. 
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